1. Field of Invention
The present invention relates generally to intercepting system calls made to multitasking operating systems, and specifically to selective interception of system calls made by specific processes;
2. Background of Invention
Multitasking operating systems such as UNIX and Microsoft Windows NT (copyright) are widely utilized in commercial computing systems. Among their many commercial uses, multitasking operating systems are commonly deployed on Internet and other network server computers. With the popularity and success of the Internet, server computer operating systems are currently of great commercial importance.
Although multitasking operating systems include various internal resources, it is often desirable to customize or extend operating system functionality for a particular use on a server computer. Such customization allows a computer programmer, a network administrator, or a webmaster to utilize the operating system in a specific manner beyond the default system capabilities provided by the manufacturer of the operating system. One method of extending and expanding operating system functionality is the interception of system calls.
A system call is a subroutine, the object code of which is located in an operating system, such that the subroutine can be called by processes executing under the control of the operating system. When executed, a system call performs some system operation, such as the access of a system hardware or software resource. Examples of operations executed by system calls include reading data from a file, opening a network communication channel, and allocating computer memory to a specific process. Application programs (processes) executing under the control of the operating system call a subroutine (make a system call) in order to bring about the performance of these and other system operations.
In order to make a system call, arguments are programmatically loaded into specific registers of the central processing unit of the computer on which the operating system is executing. One of these arguments identifies the specific system call that is being made. This argument is typically in the form of a number that is an offset into the operating system interrupt vector table, which contains pointers to the actual executable code of the system calls. The other loaded arguments include parameters to be passed to the system call.
Once the arguments have been loaded, a software interrupt is generated, signaling to the operating system that a process is requesting execution of a system call. The operating system reads the registers, and executes the requested system call with the specified parameters. The system call executes and performs the desired functionality. If the system call generates a return value, it places the generated return value (or a pointer thereto) in a pre-designated register where it can be accessed by the calling process.
In order to intercept a system call, a pointer in an interrupt vector table to a system call is replaced with a pointer to alternative object code to be executed instead of the system call. Then, when the system call is made, the alternative object code will execute instead. The alternative object code is known as a system call wrapper.
The interception of system calls is useful to extend and customize operating system functionality. For example, the interception of system calls can be used to manipulate operating system access privileges to provide security beyond that which is provided by the operating system. Through the interception of system calls that provide access to the file system, processes can be prevented from manipulating files. This is desirable, for example, when a user wishes to remotely execute a program residing on a web server, but does not want the remote program to be able to read or alter private data on the user""s computer. Today, Java applets are commonly employed to provide such security. However, many programs which users wish to remotely execute are written in languages other than Java. System call interception allows programs written in any language to be safely executed remotely.
The interception of system calls in multitasking operating systems is known today, although it is an advanced systems programming technique. Multitasking operating system call interception is not widely employed in commercial programming, but select expert systems programmers utilize the technique. Nonetheless, two serious shortcomings limit the usefulness of system call interception as it is known today.
First, when a system call is intercepted, the system call wrapper is executed whenever any process executing under the control of the operating system makes the intercepted system call. No mechanism presently exists to allow selective interception of a system call by only certain processes. It would be desirable to selectively intercept system calls such that only certain processes execute the wrapper, whereas other processes execute the default system call. For example, if file system access calls are intercepted as described above, no processes will be able to access the standard file system calls. Although it is desirable for remotely executed processes to be so restricted, this may not be the case for many local processes which should be allowed access to the file system without restriction. Thus, it is desirable to have a method whereby system calls could be selectively intercepted such that only select processes execute the system call wrapper when a system call is made.
Another shortcoming with current system call interception technology is difficulty of development. System call wrappers are inserted into the operating system, usually by loading a module into an active operating system kernel. Thus, system call wrappers execute in a part of computer memory reserved for the operating system (operating system address space).
System call wrappers, like all computer programs, require extensive testing and debugging during the development cycle. When a computer program is being developed and tested, it inevitably generates execution errors and performs illegal instructions many times before it is debugged and complete. Often, this results in the computer program becoming xe2x80x9clocked upxe2x80x9d because it has overwritten some of its own control memory, or because it is executing an infinite loop, or the like.
Normally, multitasking operating system application programs execute in an area of computer memory reserved for non-system processes (user address space). Each program (process) is assigned, by the operating system, a private block of computer memory in user address space in which it can execute. This block of memory is known as the process address space of the associated process. Therefore, when a program generates execution errors during development, other processes, and the operating system itself, are not effected. Even if an individual process locks up, other processes continue to execute, and operating system resources can be utilized to terminate the locked process so that development may continue.
System call wrappers execute in operating system address space, and therefore system call wrapper execution errors effect the entire system. When a wrapper generates execution errors, it can overwrite operating system resources such as the interrupt vector table or memory map. This requires that all processes executing under control of the operating system be terminated, and that the operating system be restarted. This has the potential to be extremely costly, as processes can be executing important commercial functionality (i.e. bank wire transfers) at the time the system terminates. Even where no critical data is lost, system downtime is expensive and undesirable.
What is needed is a method by which system calls are selectively intercepted such that the system call wrapper only executes when a system call is made by select processes. When the system call is made by a non-select process, the default system call is executed. Additionally, a method by which system call wrappers execute in process address space would be desirable, in order to avoid the expense and other hazards associated with executing system call wrappers in operating system address space.
The present invention allows the selective interception of systems calls by specific processes. Additionally, the present invention allows a system call wrapper to execute in process address space of computer memory.
In one preferred embodiment, an interception module is loaded into the operating system. Pointers in the interrupt vector table to system calls to be intercepted are replaced with pointers to the interception module. Select processes that are to intercept system calls are loaded into process address space by a modified loader program.
A loader program is an operating system utility that is used to execute computer programs that are stored on static media. Typically, a loader program loads an executable image from static media into process address space, and then initiates execution of the loaded image by transferring execution to the first instruction thereof.
Like a standard loader program, the modified loader of the present invention loads executable images from static media into process address space. Additionally, the modified loader loads an initialization module and a system call wrapper into the process address space of the loaded executable image. Each select process that is to intercept system calls is loaded by the modified loader program, whereas non-selected processes are loaded with a standard loader.
Rather than executing the loaded image itself, the modified loader executes the loaded initialization module. The initialization module registers an entry point in the system call wrapper with the interception module. The interception module maintains an association table of the select processes and system call wrapper entry points In this fashion, the interception module can determine the entry point in each system call wrapper unique to each user process.
In an alternative embodiment, a modified loader program is utilized to load both selected and non-selected processes. In that embodiment, a list of selected processes is stored in computer memory. The loader utilizes the list to determine if a process to be loaded is selected. If so, the modified loader program loads the process, the system call wrapper, and the initialization module, as described above. If the process is not selected, the loader simply loads the process in the manner of the default loader program.
When an intercepted system call is made by a process, the operating system uses the pointer in the interrupt vector table to execute the interception module (non-intercepted system calls execute normally). The interception module determines from the association table whether the calling process is one of the selected processes with a registered entry point in a system call wrapper. If so, the interception module prepares to call the appropriate system call wrapper.
Before calling the system call wrapper, the interception module first determines whether the system call was made by the wrapper, so as to avoid unwanted system call wrapper recursion. System call wrappers, like other processes, can make system calls. When a wrapper makes a system call, the interception module proceeds to call the system call, not to recursively call the wrapper.
If the system call was not made by the wrapper, the interception module stores an address to which execution control is to be returned once execution of the system call wrapper has completed. Then, the interception module proceeds to call the system call wrapper, which executes in the process address space of the calling process.
If it is determined that the process that called the system call is not one of the processes which have been selected to intercept system calls, the interception module transfers execution to the default system call, which executes as if it had been called by the process directly.
Thus, the present invention provides both selective interception of system calls by specific processes and execution of system call wrappers in process address space. Only select processes are executed by the modified loader, and thus only these select processes have a system call wrapper loaded into their process address space. When a system call is made by one these select processes, the interception module calls the system call wrapper with which the process is associated. When system calls are made by non-select processes, the interception module calls the default system call. Therefore, a system administrator of a multitasking operating system can determine which select processes will intercept which select system calls, thereby overcoming the above described limitation of the prior art.
Furthermore, the present invention overcomes the hazards associated with executing system call wrappers in operating system address space. Because the system call wrappers execute in process address space, system call wrapper execution errors do not effect other processes or the operating system itself. This results in an ease of development and a level of operating system stability heretofore unavailable using system call wrappers.
In another embodiment of the present invention, system call wrappers are loaded into user address space, but not into the process address space of any specific process. In this embodiment, when a process is loaded into memory the modified loader program loads a system call wrapper into user address space and executes the loaded wrapper. The wrapper registers an entry point in itself with the interception module. The interception module updates the association table to include an association between the process being loaded and the registered system call wrapper entry point. The modified loader then loads and executes the process. When the process makes a system call, the operating system uses the pointer in the interrupt vector table to execute the interception module. The interception module determines from the association table whether the calling process is one of the selected processes with a registered entry point in a system call wrapper. If not, the calling process is not one of the processes which have been selected to intercept system calls, so the interception module makes the default system call for the process. If so, the interception module calls the wrapper.
This embodiment of the present invention also overcomes the failings of the prior art. Only select processes are loaded by the modified loaded program, and thus only select processes are associated with a system call wrapper. This facilitates selective interception of system calls by specific processes only. Furthermore, the system call wrappers execute in user address space, thereby overcoming the hazards associated with operating system address space execution of system call wrappers.
In another embodiment, a system call wrapper executes in operating system address space, but system call interception is still selective. Interrupt vector table pointers to system calls to be intercepted are replaced with pointers to the system call wrapper, and the original pointers to the system calls are saved. The system call wrapper maintains an identifier table that indicates which processes have been selected to intercept which system calls. When a system call is made, the wrapper executes and determines whether the calling process has been selected to intercept the system call that was made. If so, the wrapper executes. Otherwise, the wrapper utilizes the saved pointer to make the default system call. This embodiment also provides for system call interception selectivity.